AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	auc	0.5	0.95
	2_DecisionTree	Decision Tree	auc	0.51616	7.94
	3_Default_LightGBM	LightGBM	auc	0.5369	3.29
	4_Default_Xgboost	Xgboost	auc	0.560548	61.95
	5_Default_CatBoost	CatBoost	auc	0.55687	8.54
	6_Default_NeuralNetwork	Neural Network	auc	0.515371	9.29
	7_Default_RandomForest	Random Forest	auc	0.533887	14.74
	8_Default_ExtraTrees	Extra Trees	auc	0.531959	10.73
the best	Ensemble	Ensemble	auc	0.566304	0.75

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 5_Default_CatBoost

<< Go back

CatBoost

• n_jobs: -1
• learning_rate: 0.1
• depth: 6
• rsm: 1
• loss_function: Logloss
• eval_metric: AUC
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

7.7 seconds

Metric details

	score	threshold
logloss	0.69495	nan
auc	0.55687	nan
f1	0.657266	0.120183
accuracy	0.548716	0.502599
precision	0.578947	0.681924
recall	1	0.120183
mcc	0.0962618	0.460723

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.69495	nan
auc	0.55687	nan
f1	0.520012	0.502599
accuracy	0.548716	0.502599
precision	0.542395	0.502599
recall	0.499404	0.502599
mcc	0.0958501	0.502599

Confusion matrix (at threshold=0.502599)

	Predicted as 0	Predicted as 1
Labeled as 0	1043	707
Labeled as 1	840	838

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 8_Default_ExtraTrees

<< Go back

Extra Trees Classifier (Extra Trees)

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

9.7 seconds

Metric details

	score	threshold
logloss	0.691503	nan
auc	0.531959	nan
f1	0.657266	0.339889
accuracy	0.526838	0.513933
precision	0.54902	0.524412
recall	1	0.339889
mcc	0.0563493	0.485714

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.691503	nan
auc	0.531959	nan
f1	0.323603	0.513933
accuracy	0.526838	0.513933
precision	0.538889	0.513933
recall	0.231228	0.513933
mcc	0.0509463	0.513933

Confusion matrix (at threshold=0.513933)

	Predicted as 0	Predicted as 1
Labeled as 0	1418	332
Labeled as 1	1290	388

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

0.3 seconds

Metric details

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Confusion matrix (at threshold=0.440414)

	Predicted as 0	Predicted as 1
Labeled as 0	0	1750
Labeled as 1	0	1678

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_LightGBM	1
4_Default_Xgboost	1
5_Default_CatBoost	1

Metric details

	score	threshold
logloss	0.68698	nan
auc	0.566304	nan
f1	0.657266	0.244451
accuracy	0.551925	0.511812
precision	0.646341	0.639755
recall	1	0.244451
mcc	0.104373	0.443417

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.68698	nan
auc	0.566304	nan
f1	0.497711	0.511812
accuracy	0.551925	0.511812
precision	0.551449	0.511812
recall	0.453516	0.511812
mcc	0.10173	0.511812

Confusion matrix (at threshold=0.511812)

	Predicted as 0	Predicted as 1
Labeled as 0	1131	619
Labeled as 1	917	761

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

7.0 seconds

Metric details

	score	threshold
logloss	0.694754	nan
auc	0.51616	nan
f1	0.657266	0.31903
accuracy	0.517211	0.529372
precision	0.621622	0.535765
recall	1	0.31903
mcc	0.0281982	0.465613

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.694754	nan
auc	0.51616	nan
f1	0.378053	0.529372
accuracy	0.517211	0.529372
precision	0.511699	0.529372
recall	0.299762	0.529372
mcc	0.0281597	0.529372

Confusion matrix (at threshold=0.529372)

	Predicted as 0	Predicted as 1
Labeled as 0	1270	480
Labeled as 1	1175	503

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 3_Default_LightGBM

<< Go back

LightGBM

• n_jobs: -1
• objective: binary
• num_leaves: 63
• learning_rate: 0.05
• feature_fraction: 0.9
• bagging_fraction: 0.9
• min_data_in_leaf: 10
• metric: auc
• custom_eval_metric_name: None
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

2.3 seconds

Metric details

	score	threshold
logloss	0.690965	nan
auc	0.5369	nan
f1	0.657266	0.344372
accuracy	0.541132	0.501621
precision	0.614907	0.533716
recall	1	0.344372
mcc	0.0801263	0.501621

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.690965	nan
auc	0.5369	nan
f1	0.431925	0.501621
accuracy	0.541132	0.501621
precision	0.548121	0.501621
recall	0.356377	0.501621
mcc	0.0801263	0.501621

Confusion matrix (at threshold=0.501621)

	Predicted as 0	Predicted as 1
Labeled as 0	1257	493
Labeled as 1	1080	598

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 6_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

8.5 seconds

Metric details

	score	threshold
logloss	1.1384	nan
auc	0.515371	nan
f1	0.657266	0.350824
accuracy	0.523337	0.389805
precision	0.552381	0.694902
recall	1	0.350824
mcc	0.0470049	0.694902

Metric details with threshold from accuracy metric

	score	threshold
logloss	1.1384	nan
auc	0.515371	nan
f1	0.228517	0.389805
accuracy	0.523337	0.389805
precision	0.55	0.389805
recall	0.144219	0.389805
mcc	0.046444	0.389805

Confusion matrix (at threshold=0.389805)

	Predicted as 0	Predicted as 1
Labeled as 0	1552	198
Labeled as 1	1436	242

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 7_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

13.8 seconds

Metric details

	score	threshold
logloss	0.691282	nan
auc	0.533887	nan
f1	0.657266	0.379034
accuracy	0.53063	0.496192
precision	0.545156	0.509073
recall	1	0.379034
mcc	0.0579182	0.496192

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.691282	nan
auc	0.533887	nan
f1	0.459886	0.496192
accuracy	0.53063	0.496192
precision	0.526518	0.496192
recall	0.408224	0.496192
mcc	0.0579182	0.496192

Confusion matrix (at threshold=0.496192)

	Predicted as 0	Predicted as 1
Labeled as 0	1134	616
Labeled as 1	993	685

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

61.0 seconds

Metric details

	score	threshold
logloss	0.707584	nan
auc	0.560548	nan
f1	0.657266	0.0820577
accuracy	0.54755	0.572514
precision	0.653846	0.773927
recall	1	0.0820577
mcc	0.0945318	0.572514

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.707584	nan
auc	0.560548	nan
f1	0.428308	0.572514
accuracy	0.54755	0.572514
precision	0.561353	0.572514
recall	0.346246	0.572514
mcc	0.0945318	0.572514

Confusion matrix (at threshold=0.572514)

	Predicted as 0	Predicted as 1
Labeled as 0	1296	454
Labeled as 1	1097	581

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back